1. Field of the Invention
The present invention relates generally to handwriting data entry, and more particularly to methods and systems for facilitating data entry using a pen input device.
2. Description of the Related Art
Digital writing instruments, interchangeably referred to herein as xe2x80x9cdigital pensxe2x80x9d regardless of whether they write in ink, can be used to capture pen strokes on paper and digitize them, so that the pen strokes can be converted by handwriting recognition software to a digitally-stored record of the writing. In this way, a laborious, tedious step in modern written communication, namely, the manual transcribing of handwriting into a computerized word processor, is eliminated, greatly increasing productivity.
Accordingly, digital pen systems can digitize pen strokes across a substrate, by sensing, in some fashion, the time-dependent position of the pen and converting the positions to pen strokes, for input of digital representations of the pen strokes to a handwriting recognition device. As recognized herein, many previous systems require special pads to sense pen or stylus motion. As further recognized herein, it is desirable to enable a writer to write characters on a conventional paper substrate while recording pen motion without requiring the use of a special underlying pad, so that the user can write on any paper substrate without having to transport and use a relatively bulky pad. The present invention moreover recognizes that it is desirable to provide a system that can, but not must, use special paper for particular applications.
In the present inventor""s and present assignee""s co-pending U.S. patent application Ser. No. 09/557,579, filed Apr. 22, 2000, a digital pen device is disclosed in which a laser is mounted on the pen to direct coherent light against a paper substrate. Reflected interference patterns, referred to as speckles, are imaged and processed to determine relative pen movements. Special (bar-coded) paper can be used to anchor the relative pen movements to absolute locations. The present invention seeks to extend the principles disclosed in the ""579 application to providing a pen that uses non-coherent light for facilitating use of a single pen with plain paper, quad paper, and special paper having any one of a number of bar code schemes, without requiring the relatively small processor of the pen to know the particular kind of paper being used or to know any particular bar code pattern when special paper is used.
A xe2x80x9cmemo penxe2x80x9d is disclosed by Nabeshima et al. in CHI ""95 Proceedings. The xe2x80x9cmemo-penxe2x80x9d is an ordinary pen with the exception that is takes images of what it draws and then xe2x80x9cstitchesxe2x80x9d the images together to produce a mosaic of the paper. As recognized by the present invention, the device of Nabeshima et al. requires taking and saving to memory a full image each processing cycle, with the relatively large number of images saved to onboard non-volatile memory in turn dictating that each image size be relatively small. Even so, according to Nabeshima et al. so much memory is consumed that only about fifteen minutes of handwriting can be stored in the pen. The present invention recognizes the above-stated drawbacks, and furthermore recognizes the desirability of providing a pen that can image bar-coded paper without requiring the pen to know any bar codes, thus enabling the use of a relatively simple, low cost pen input device.
A digital pen includes an elongated body defining a writing tip. A light source is oriented on the body to direct light onto a substrate against which the writing tip can be positioned to deposit a substance thereon. Also, a detector receives reflections of light from the substrate, and a pen processor receives signals from the detector and in response thereto outputs delta signals that represent differences between successive positions of the pen. The processor does this by executing a bitwise XOR of a current frame of pixels against a previous frame of pixels at plural potential positions, with a delta position being based on the potential position having a highest XOR sum. Consequently, each and every frame image need not be saved to non-volatile memory onboard the pen.
In a preferred embodiment, the reflections detected by the detector are from the substance or by the substrate. Also, a non-volatile memory is supported by the body, and the memory stores signals from the pen processor. The signals can be retrieved from the memory and processed by a handwriting recognition module to render signals representing alpha-numeric characters.
As disclosed in detail below, a contact sensor preferably is supported by the body. The contact sensor outputs a contact signal representative of whether the writing tip is positioned against the substrate. The pen processor selectively sends delta signals to the memory based at least in part on the contact signal.
In addition to generating delta signals, the pen processor periodically stores key frames. Each key frame is an image of the substrate. In greater detail, the pen processor has a local memory, and when a confidence threshold is reached as defined by a comparison between the current frame and last frame, a delta signal is stored in the non-volatile memory when the contact signal indicates the writing tip is positioned against the substrate. Otherwise, the delta signal is temporarily stored in the local memory of the pen processor. When the contact signal indicates the writing tip is positioned against the substrate and the confidence threshold is reached, a delta signal is generated every cycle and a key frame is imaged and stored only once every xe2x80x9cNxe2x80x9d cycles, wherein xe2x80x9cNxe2x80x9d is an integer greater than one. In contrast, when the contact signal indicates the writing tip is positioned against the substrate and the confidence threshold is not reached for a low confidence cycle, a key frame is imaged and stored for the low confidence cycle. In addition, a key frame is imaged and stored each time the pen is lifted and placed on the substrate. In the presently preferred embodiment, a host processor receives signals from the pen processor and accesses a library of optical patterns to determine a type of optical pattern on the substrate.
In another aspect, a digital input device includes an elongated body defining a distal tip configured for depositing a substance onto a substrate, and a light source oriented on the body to direct light toward the vicinity of the distal tip. The device also includes a detector oriented on the body to receive reflections of light source light from a substrate against which the distal from time to time can be positioned. A contact sensor is on the body, and a pen processor is also supported by the body. The processor communicates with the detector and the contact sensor to determine, each cycle period, whether to generate delta signals and to store in non-volatile memory onboard the body each update period greater than a cycle period, a key frame image of the substrate.
In still another aspect, a computer-implemented method for digitizing strokes of a writing tip across a substrate includes detecting reflections from a substrate each cycle period, and comparing a frame of reflected light in a second cycle to a frame of reflected light in a first cycle immediately preceding the second cycle to generate a delta signal. The method also includes occasionally but not every cycle generating at least one key frame, and storing the key frames and selectively storing delta signals in non-volatile memory in a pen body associated with the writing tip. The key frames and delta signals are transferred to a host processor, which is used to correlate the delta signals and key frames to alpha-numeric characters.
The details of the present invention, both as to its structure and operation, can best be understood in reference to the accompanying drawings, in which like reference numerals refer to like parts, and in which: